Evaluating the learning effectiveness of using simulations in software project management education: results from a twice replicated experiment

The increasing demand for software project managers in industry requires strategies for the development of management-related knowledge and skills of the current and future software workforce. Although several educational approaches help to develop the necessary skills in a university setting, few empirical studies are currently available to characterise and compare their effects. This paper presents the results of a twice replicated experiment that evaluates the learning effectiveness of using a process simulation model for educating computer science students in software project management. While the experimental group applied a System Dynamics simulation model, the control group used the well-known COCOMO model as a predictive tool for project planning. The results of each empirical study indicate that students using the simulation model gain a better understanding about typical behaviour patterns of software development projects. The combination of the results from the initial experiment and the two replications with metaanalysis techniques corroborates this finding. Additional analysis shows that the observed effect can mainly be attributed to the use of the simulation model in combination with a web-based role-play scenario. This finding is strongly supported by information gathered from the debriefing questionnaires of subjects in the experimental group. They consistently rated the simulation-based role-play scenario as a very useful approach for learning about issues in software project management. q 2003 Elsevier B.V. All rights reserved.